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(57) Abstract 

An automated visual inspection 
system and process are disclosed. The 
system (10) includes an imaging sys- 
tem (12) operable to provide image data 
and a precision positioning system (14) 
operable to move the imaging system 
(12) to scan an object (16). A pro- 
cessing and control engine is coupled 
to receive the image data and to pro- 
vide control signals to the precision po- 
sitioning system (14) and other com- 
ponents. The processing and control 
engine pre-processes the image data, 
represents the image using descriptors, 
compares the descriptors to information 
in a knowledge base and detects and 
classifies defects based upon results of 
the comparison. The processing and 
control engine also associates a confi- 
dence level with the classification of the 
defect. In one embodiment, if the confi- 
dence level is not above a specified tol- 
erance, the processing and control en- 
gine alerts an operator and provides a 
best option as to the classification of 
the defect, accepts a confirmation or al- 
ternate classification from the operator, 
and adds information to the knowledge 
base after a decision is made. In an- 
other embodiment, the imaging system (12) provides structured, reflected light intensity as part of the image data, and the processing and 
control engine recovers three-dimensional spatial information based upon the reflected light intensity. 
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AUTOMATED .VISUAL INSPECTION SYSTEM AND PROCESS 
FOR DETECTING AND CLASSIFYING DEFECTS 

TECHNICAL FIELD OF THE INVENTION 

This invention relates in general to automated 
inspection of manufactured products, and more 
particularly to an automated visual inspection system 
and process for detecting and classifying defects. 
BACKGROUND OF THE I NVENTION 

The need for automated inspection of manufactured 
products arises from inherent inefficiency of human 
inspectors caused, in part, by an inability to maintain 
a continuous focus and to apply a consistent analysis 
from day to day. Conventional automated inspection 
technology typically sacrifices classification 
accuracy -- and hence the ability to provide meaningful 
process control information - for detection ability and 
ease of implementation. Thus, while conventional 
inspection technologies may be able to detect a wide 
range of anomalies, they do not provide adequate 
classification of defects. This is problematic because 
detection without accurate classification does not 
provide the information needed to improve the 
manufacturing process and increase production yields. 

For example, the manufacture of populated printed 
wiring boards for computing devices is an industry in 
which automated inspection could be used to reduce the 
number of defective board escapes (i.e., boards that 
make it out of the factory and to the customer) and 
improve the yield of the production line. There should 
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be two main goals of this automated inspection: (1) to 
detect and classify visible defects and (2) to provide 
process control information. With respect to defects, 
some are at the component level, such as missing 
components, components off -pad, reverse-polarity, wrong 
components, and foreign material. Other defects are at 
the lead level and include bent and missing leads, 
individual lead skew, solder bridges and solder joints 
with dewets or insufficient solder. With respect to 
process control, defect classification and source of 
the defect information can be used to improve the 
production process, thereby improving production 
yields, reducing the number of defective boards 
produced, and reducing the time defective boards are in 
rework . 

Conventional automated inspection technology does 
not accomplish these goals. For populated printed 
wiring boards, the conventional technology is generally 
based on a technology called templating. A template is 
an image of a component or section of populated printed 
wiring board that is assumed to be good. As boards are 
being inspected, the template is subtracted from the 
image of the same component or board section taken from 
the board under inspection. The template is divided 
into a number of different regions, each with one or 
more threshold values. The difference between the 
image under inspection and the template is compared to 
the threshold to determine if there is a potential 
defect in that region. 
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Templating as a technology for automated visual 
inspection of populated printed wiring boards has some 
benefits that, in the past, made it a viable approach. 
The approach is conceptually very simple and is 
straightforward to implement in hardware. The ease of 
implementation has led to the development of 
specialized hardware designed to process a single image 
frame very quickly. The template based method is also 
capable of detecting a number of possible errors. 
Further, when defect sizes are relatively large, there 
can be enough flexibility in the specification of the 
"important" regions around a component that limited 
classification of detected defects is possible. 

However, there are significant drawbacks as well. 
Templating technology is very difficult to extend. As 
new defects arise, a set of thresholds must be adjusted 
which often is too difficult for the typical production 
line operator or impossible in some cases, and effects 
of a change can only be determined by experimentation. 
Further, if classification is required over a range of 
defects, functional additions must be made to the 
templating decision method (e.g., the magnitude of the 
differences triggers computational functions to be run, 
and the defect classification is made on the output of 
the functions.) In addition, specialized hardware 
implementations, made on the basis of an imaging system 
field of view (FOV) and resolution capable of handling 
larger defects, require substantial design modification 
when higher resolutions are needed to distinguish 
defects . 



WO 99/16010 PCT/US98/19544 

4 

Detecting discrepancies between the board under 
inspection and a template is relatively inapplicable to 
current inspection problems. Templating inspection 
methods work well where there is very little variation 
in the images to be analyzed (and where variation alone 
is a direct indication of a defect) . Historically, 
component sizes have been large enough and production 
rates have been slow enough that templating methods 
could apply threshold values "creatively" to produce a 
functional solution. However, for current populated 
printed wiring board manufacturing, there are numerous 
variations that can produce satisfactory boards -- 
e.g., variations in solder shape, replacement generic 
parts, and minor placement differences -- where the 
variations can represent a majority of the component 
itself. These variations make it unpractical, if not 
impossible, to successfully apply templating to current 
automated inspection problems and modern production 
lines . 
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SUMMARY OF THE INVENTION 

In accordance with the present invention, an 
automated visual inspection system and process for 
detecting and classifying defects are disclosed that 
provide advantages over conventional automated 
inspection systems and methods. 

According to one aspect of the present invention, 
the automated visual inspection system and process 
include an imaging system operable to provide image 
data and a precision positioning system operable to 
move the imaging system to scan an object. A 
processing and control engine, including a scalable 
processing accelerator, is coupled to receive the image 
data and to provide control signals to the precision 
positioning system. The processing and control engine 
pre-processes the image data, represents the image 
using descriptors, compares the descriptors to 
information in a knowledge base and identifies and 
classifies a defect based upon results of the 
comparison. The processing and control engine also can 
associate a confidence level with the classification of 
the defect. In one embodiment, if the confidence level 
is not above a specified tolerance, the engine alerts 
an operator and provides a best option as to the 
classification of the defect, accepts a confirmation or 
alternate classification from the operator, and adds 
information to the knowledge base responsive to the 
confirmation or alternate classification. In another 
embodiment, the imaging system provides structured, 
reflected light intensity as part of the image data, 
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and the engine _ recovers three-dimensional spatial 
information based upon the reflected light intensity 
when the image data is pre-processed. 

A technical advantage of the present invention is 
the automatic classification of defects based upon 
comparisons with information held in a knowledge base. 

Another technical advantage of the present 
invention is the use of a line scan camera, a 
telecentric optic, and structured lighting to allow the 
inspection system to collect three-dimensional spatial 
information about a scanned object in addition to a 
two-dimensional spatial image of the scanned object. 

An additional technical advantage of the present 
invention is the modular architecture which allows for 
scaling of data collection and processing to desired 
levels of resolution, accuracy, and throughput 
(processing speed) . 

A further technical advantage of the present 
invention is the capability to adjust the physical 
platform to mechanically adjust for skew. Also, the 
system can periodically check for and remove fixed 
pattern errors caused by the system's mechanical 
characteristics . 

Another technical advantage of the present 
invention is the maintenance of information between a 
number of inspection systems about classifications of 
prior inspections to place each decision in historical 
context . 
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Additional technical advantages should be readily 
apparent to one skilled in the art from the following 
figures, descriptions, and claims. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

A more complete understanding of the present 
invention and the advantages thereof may be acquired by 
referring to the following description taken in 
5 conjunction with the accompanying drawings, in which 

like reference numbers indicate like features, and 
wherein: 

FIGURE 1 is a block diagram of one embodiment of 
an automated visual inspection system according to the 
10 present invention; 

FIGURE 2 is a flow diagram of one embodiment of a 
process for detecting and classifying defects 
implemented by the system of FIGURE 1; 

FIGURE 3 is a diagram of an example of a detected 
15 * defect mapped onto a decision space of a knowledge base 
according to the present invention; 

FIGURES 4A, 4B and 4C are diagrams of three 
possibilities for classification of the detected defect 
of FIGURE 3 according to the present invention ; 
20 FIGURE 5 is a diagram of one embodiment of an 

automated visual inspection station for detecting and 
classifying defects in printed wiring boards according 
to the present invention; 

FIGURE 6 is a diagram of one embodiment of the 
25 imaging system of the station of FIGURE 5; and 

FIGURE 7 is a flow chart of one embodiment of an 
automated inspection process implemented using the 
station of FIGURE 5. 
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DETAILED DESCRIPTIO N OF THE IN VENTION 

In general, it is not adequate for automated 
visual inspection systems to simply detect anomalies. 
Often, there are too many anomalies that are viable 
component constructions and should not be identified as 
defects. To be effective in supplying a manufacturing 
entity the information needed to improve yields on the 
production line, inspection stations need to provide 
accurate classification (both defective and non- 
defective) of anomalies and need to report defects and 
variations to the user in a way that allows predictions 
of production line performance. The weaknesses of 
templating and other conventional techniques motivated 
the development of a fundamentally different technology 
for visual inspection. According to the present 
invention, adaptive knowledge -based reasoning is used 
which stores inspection knowledge in a database and 
uses that knowledge to generate high-quality defect 
detection and classification. The benefits of this 
technology go beyond accurate classification by making 
setup, operation, and maintenance a more simple, end- 
user task. 

Automated, vjsuptl J:^spgct^Q^ System 

FIGURE 1 is a block diagram of one embodiment of 
an automated visual inspection system, indicated 
generally at 10, which is based on an adaptive 
knowledge -based reasoning technology according to the 
present invention. System 10 includes an imaging 
system 12 that is coupled to a precision positioning 
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system 14 . Precision positioning system 14 operates to 
position imaging system 12 to scan an object 16 that is 
being inspected. Object 16 is coupled to a material 
handling system 18 which operates to position object 16 
within the inspection range of imaging system 12. 
Imaging system 12, precision positioning system 14 and 
material handling system 18 are coupled to and 
communicate with a host computer system 20 which 
provides a processing and control engine for system 10. 

Material handling system 18 provides gross 
manipulation of object 16 and delivers object 16 for 
viewing by imaging system 12. Material handling system 
18 can include robotic, mechanical, or manual movement 
to position object 16. Precision positioning system 
14, which is coupled to imaging system 12, operates to 
position imaging system 12 over object 16 and to move 
imaging system 12 to scan object 16. In one 
embodiment, precision positioning system 14 sweeps the 
length of object 16, orthogonally steps over, and 
continues to sweep until the width of object 16 is 
completed by successive parallel sweeps. It should be 
understood that alternatives exist for obtaining the 
scanned image, including having a fixed camera with 
precision motion of object. 

In the embodiment of FIGURE 1, imaging system 12 
includes an illumination system 22 which operates to 
illuminate object 16 during inspection. An optic 24 
views the illuminated object 16 and provides an image 
to imager 26. Imager 26, in turn, provides image data 
to computer system 20. In one embodiment, imager 26 
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can be a time delay integration (TDI) line scan camera 
which provides relatively high resolution over a large 
field of view and increased light sensitivity. The TDI 
line scan camera operates with a continuous scan and 
provides high speed imagery without the need for 
excessive light. Optic 24 maps a given object field of 
view (FOV) onto the sensor array of imager 26 based 
upon optic 24 magnif ication/demagnif ication. In one 
embodiment, optic 24 is a telecentric optic with 
constant perspective and magnification across the 
entire FOV to achieve a top-down view of object 16 with 
relatively little parallax distortion or occlusions. 
All other fixed pattern optical distortions can then be 
mapped out by calibration algorithms running on host 
computer 20. Illumination system 22 can implement 
various types of lighting. Lighting approaches can 
include, for example, diffuse, back lighting, polarized 
front illumination, and directional structured front 
illumination. Different light sources provide 
different spectrums of light each being useful in 
different situations. In general, the illumination 
system 22 needs to be tailored to match the desired 
task. In one embodiment, a white light source with a 
bifurcated fiber optic light array that provides a 
structured, focused, off -axis light illuminating object 
16. This illumination allows three-dimensional spatial 
information to be acquired based upon the intensity of 
the reflected light received through the optic 24. 

Computer system 20 includes control hub 28 which 
interfaces between computer system 20 and various 
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connected components. Host computer system 20 can be 
an IBM compatible personal computer, with typical 
components, which is rack mounted. Computer system 20 
executes software engine 30 which provides a user 
interface and performs the classification analysis of 
image descriptions. Control hub 2 8 receives status 
signals from the connected components and provides the 
status signals to software engine 30. Software engine 
processes the status signals, in addition to the image 
descriptors, and generates control signals to manage 
operation of the components. The control signals are 
then provided to the various components through control 
hub 28. As shown, precision positioning system 14, 
material handling system 18, illumination system 22, 
and optic 24 can all receive control signals and 
provide status signals through control hub 28. In one 
embodiment, control hub 28 can include optical 
interfaces (e.g., OPTO 22) for digital signals and 
digital /analog (D/A) interfaces for analog signals. 

Computer system 2 0 further includes a data 
input/output (I/O) card 32 that controls and receives 
the image data from imager 26. Data I/O card 32 can 
function essentially as a frame grabber which 
reconstructs the scanned image and converts the data 
for processing accelerator card 34. Processing 
accelerator card 34 processes the image data to provide 
image descriptors to software engine 30. Processing 
accelerator card 34 performs image processing 
algorithms on the image data to produce the descriptors 
of the image. Processing accelerator card 34 then 
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provides the image descriptors to software engine 30 
which performs defect detection and classification by 
comparing the descriptors to information in a knowledge 
base of historical information. 

Computer system 20 further includes support 
interface hardware 36 which interface between computer 
system 20 and network devices and/or peripherals 38, 
such as a printer, back-up drives, networking devices, 
electronic message (e.g., email, TCP/IP, paging, or 
other messages) interface, keyboard and monitor. 
Control hub 28 can also provide control signals to a 
system platform 39 which provides a physical support 
for system 10. In addition, platform 39 can be 
manually adjusted to fine tune system 10 and compensate 
for error introduced by the physical characteristics of 
platform 39 or of the other components. 

Detection and Classification 

In general, software engine 3 0 references 
populations of information about defects stored in its 
knowledge base. When software engine 30 makes a 
decision, it associates a confidence level with the 
decision. Software engine 30 typically has more 
confidence when a defect is similar to prior defects 
and a number of similar defects have been seen. The 
analysis is essentially an adaptive, 
experience/knowledge -based correlation with an 
underlying statistical basis or behavior. The 
practical issues for confidence in a decision can 
include how many examples of the defect, how many times 
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a classification has been done in a certain way, how 
similar the defect is to other defect classes in the 
knowledge base, relationship to other current defect 
classifications (defects found in other board 
5 locations) , and what operator, if any, has entered the 

classification. Once determined with sufficient 
confidence, classification of defects allows the 
manufacturing entity not only to correct the defects 
but also to identify and correct the source of the 

10 problem. 

FIGURE 2 is a flow diagram of one embodiment of 
the process for detecting and classifying defects 
implemented by system 10 of FIGURE 1. In region 40, an 
original image of object 16 is obtained from imager 26. 

15 The image data is transformed, in region 42, into a set 

of symbolic and real -valued descriptors by data I/O 
card 32 and processing accelerator card 34. This 
trans format ion can be accomplished through the use of 
computer aided design (CAD) component location and 

20 characteristic information, rules for determining what 

set of preprocessing routines need to be run on the 
image, and image processing routines for generating 
descriptor values. In region 44, the computation and 
aggregation of the descriptor values can be viewed as a 

25 re-description of the image, previously represented as 

a set of pixels, now represented as a set of 
descriptor-value pairs or other appropriate 
descriptors. The descriptors need not be a complete 
description of the image if the set of descriptors are 

30 chosen so that they provide the information necessary 
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to determine If a defect exists, and, if it does, to 
what class of defects it belongs. In other words, the 
descriptors reflect the semantics of the image and 
inspection process. 

In region 46, system 10 performs historical 
similarity based classification using an adaptive 
knowledge base of historical information. The 
knowledge base provides a general, wide breadth of 
information about the inspection and classification of 
assembly components. This knowledge is stored in such 
as way as to permit a historical similarity-based 
classification decision making methodology to be 
employed during decision making. This method of 
classification uses the knowledge base of general 
inspection knowledge which is augmented with specific 
knowledge of how defects appear on different components 
of a given object 16. 

Decision' Space 

FIGURE 3 is a diagram of an example of a detected 
defect mapped onto a decision space of the knowledge 
base according to the present invention. One way to 
represent the tree- like structure depicted at region 46 
of FIGURE 2 is as a multi -dimensional space. This 
multi-dimensional space reflects the partitioning of 
knowledge into defect regions that correspond to either 
a defect class or to a prototypical defect. In the 3D 
example of FIGURE 3, there are two known defects having 
defined defect regions 50 and 52, as shown. 
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Suppose that a new defect point 54 is presented 
after inspection which does not fall within either 
region 50 or region 52. The inspection system thus 
needs to determine how to classify point 54 . Because 
point 54 is outside of the decision boundaries of 
regions 50 and 52, it may not be possible to say with 
confidence that the point 54 is a member of either 
class. Looking at class information, such as the 
"spread" or spatial coverage of the class, the 
inspection system may prefer defect class 1 (region 50) 
over defect class 2 (region 52) , defect class 2 over 
defect class 1, or neither. 

FIGURES 4A, 4B and 4C are diagrams of three 
possibilities for classification of the detected defect 
of FIGURE 3 according to the present invention. As 
shown in FIGURE 4A, one alternative is that point 54 
should be classified as a member of defect 1. In this 
case, the decision space is modified to expand defect 
region 50, as shown in FIGURE 4A. A second alternative 
is shown in FIGURE 4B in which point 54 is classified 
as a member of defect 2, and defect boundary 52 is 
expanded. A third alternative is shown in FIGURE 4C in 
which point 54 is classified as a member of a new 
defect class 3 defined by a new region 56. This 
determination of how to classify point 54 into a defect 
class can be made by taking into account the relative 
similarity between the new point 54 and defect classes 
1 and 2. Also, in cases where a high confidence 
classification cannot be made, the operator can be 
solicited for an opinion. The information gained from 
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the operator can then be used to determine which 
alternative is selected and to provide important 
feedback to the system for future classifications. 

Actgiptive ifoQwiQticre Base3 Reasoning 

The use of adaptive knowledge based reasoning for 
detection and classification of product defects 
provides a number of advantages over conventional 
approaches to automated vision and inspection 
applications. One advantage to the process used in the 
present invention is that images are represented in 
semantic space by the use of descriptors- Rather than 
work directly with image pixels (syntactic space) or 
with simple geometric descriptions of images (symbolic 
space) , the inspection system represents information 
contained in the image with semantically meaningful 
descriptors (semantic space) . The descriptors embody 
selected descriptive components of the image which 
allow the inspection system to distinguish a defect 
class or set of classes from others. 

A second advantage is that defects can be 
arbitrarily close to one another in the defect space. 
In ternplating systems, one of the difficult things to 
resolve is where two defect classes are visually 
similar. The inspection system of the present 
invention, on the other hand, allows defect classes to 
be arbitrarily close to one another as long as there is 
at least one feature by which they can be 
discriminated . 



WO 99/16010 



PCT/US98/19544 



18 

Another advantage is that new information on 
defect classification can be introduced at any time 
during system operation. For example, as shown in 
FIGURES 4A, 4B and 4C, a new data point or defect can 
be added that modifies the boundary between two defect 
classes. The addition of this new knowledge can be 
used to refine the decision boundary between classes to 
an arbitrarily fine degree. 

A fourth advantage is the introduction of 
knowledge into the knowledge base is additive, meaning 
that members of other existing defect classes are not 
reclassified simply because one defect class is being 
modified by the introduction of new information. This 
is unlike tetnplating methods, where the modification of 
a threshold value is likely to upset the classification 
accuracy of a number of defect classes and the total 
effects can only be determined by experimentation 
(across multiple board types and lots) . 

A further advantage of the present knowledge base 
is that decision making is explainable, justifiable, 
and correctable. The decision making component of the 
inspection system enables any classification made by 
the system to be presented to the operator with the 
basis for the decision. Often, the explanation is 
based on previous interactions with the operator, 
enabling the growth of trust in the operation of the 
inspection system. 

Another advantage is that the inspection system is 
designed in an object-oriented, modular fashion, so 
that the speed at which the system runs can be 
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regulated by the type of computational support given to 
it. For example, inspection times in the range of two 
minutes to below twenty seconds can be realized for a 
typical twenty by eighteen inch printed wiring board. 

The knowledge based decision making for automated 
visual inspection of products, such as populated 
printed wiring boards, has the potential to fill in 
many information gathering holes in current assembly 
production processes. The present technology can 
detect and classify relevant defects by making 
informed, reasoned judgments on the components in 
images. The system can do this because it works with 
semantic descriptors of images that are designed to 
maximize discrimination between defect classes, and 
because the system has the knowledge to make those 
decisions with regard to other possible defects . 

The present technology also can provide accurate 
process control information. Accurate defect 
classification information is an important requirement 
for providing accurate process control information. 
Detection of anomalies is generally not enough to 
generate useful process control information. Further, 
conventional defect detection systems often rely 
completely on human operators to make classification 
decisions which results in data that is not timely or 
consistently accurate. 

The present invention reduces false alarms. The 
ability to accept new information from the operator in 
real-time allows the system to reduce the number of 
false alarms that it produces, and eliminates the 
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reoccurrence of any particular false alarm. The 
knowledge structuring and decision making features of 
the system allow it to learn the correct classification 
of a defect that was initially classified incorrectly 
and not repeat that error. The corrected 
classification becomes part of the knowledge base which 
is retained from run to run, applying the learned 
knowledge to a wide variety of new component instances . 

The system also simplifies programming of new 
boards for inspection. Populated printed wiring board 
inspection is made significantly easier by this 
adaptive learning technology. Bringing a never-before- 
seen board through inspection programming can be done 
by presentation of CAD information and a sample board 
(populated and unpopulated) and through selection of 
inspection criteria preferences. The present system 
also gives the manufacturer an ability to perform full 
100% inspection of boards assembled on an equipped 
production line. This can be viewed in two ways: 1) it 
provides a gate facility that prevents defective boards 
from escaping the manufacturing facility; and 2) it 
provides continuous monitoring feedback that allows new 
insights into the production process to be found and 
exploited . 

Inspection Station 

FIGURE 5 is a diagram of one embodiment of an 
automated visual inspection station, indicated 
generally at 60, for detecting and classifying defects 
in printed wiring boards according to the present 
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invention. Station 60 includes a platform 62 and 
vibration isolated subplatf orm cradle 63 constructed 
from metal components to provide physical support for 
station 60. A pair of guides 64 are mounted on 
subplatform 63 and provide a path for populated printed 
wiring boards 66. A stop 68 can be used to stop each 
board 66 at the appropriate position for inspection, 
and a thumper 69 can be used to position board 66 
against one of guides 64. Guides 64, stop 68 and 
thumper 69 form a material handling system for boards 
66. This material handling system can be 18 inches 
wide by one meter long with a two speed, reversible 
conveyor to handle boards 66 and can have a SMEMA 
interface built into platform 62 for interfacing with 
other components of a manufacturing line. 

Station 60 further includes a light source 70 
which provides light for illuminating board 66. Light 
source 70 can be a FOSTEC DC regulated 150 Watt source 
with analog control and long life quartz Halogen bulbs 
(DDL type) . Station 60 also includes a camera 72 that 
receives an image through an optic 74 . Camera 72 can 
be a DALSA CL-E2 2048A TDI line scan camera with a 
VISION 1 PS1 linear power supply for CL-E2. Optic 74 
can be an INVARITAR telecentric optic with .37x 
magnification over a 72 millimeter field of view (FOV) . 
As shown, the light generated by light source 70 is 
split to two heads 76 through a bifurcated optical 
fiber 78. The heads 76 can be FOSTEC 6 inch line heads 
with cylindrical lenses. Light source 70, camera 72, 
optic 74, heads 76 and optical fiber 78 together form 
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an imaging system for scanning and acquiring images of 
board 66. Because of the structured directed light 
provided by heads 76, three-dimensional spatial 
information can be extracted/computed from the 
intensity of the reflected light during scans. FIGURE 
6 is a diagram showing more detail of one embodiment of 
this imaging system. 

Station 60 has a precision positioning system 
associated with the imaging system. The precision 
positioning system includes a threaded precision guide 
80 having a motor 82 and a threaded precision guide 84 
having a motor 86. (In other implementations, guides 
84 and 86 could be, for example, pneumatic, hydraulic 
or magnetic.) Motors 82 and 86 include motor drive 
units which can be simultaneously controlled by a two- 
axis controller in interface devices 88 . These 
components allow the imaging system to be moved across 
board 66 to scan in an image of board 66. The threaded 
precision guides 80 and 84 can be DAEDAL 406 series 
linear stages (e.g., 16 and 20 inch) with precision 
linear encoders and two pitch lead screws. Motors 82 
and 86 can be two ZETA motor drives (one per axis) 
controlled by COMPUMOTOR 6200 Series Two axis indexers . 

Interface devices 88, which include optical relays 
for digital control signals and digital /analog 
converters for analog control signals, interface 
between a computer system 90 and the various controlled 
components. Computer system 90 provides a processing 
and control engine for operating inspection station 60 
and operates generally in the same manner as described 
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above with respect to FIGURE 1 . Computer system 90 
receives status information and provides control 
signals as well as communicating with peripherals 92. 
Computer system 90 can be an Industrial Computer Source 
5 rack mounted chassis having a single board computer 

with an INTEL PENTIUM PRO 200 MHz processor and other 
typical computer components. Interface devices 88 can 
include a 6200 indexer interfacing through a standard 
RS232 serial port and computer boards and digital -to- 

10 analog and OPTO 22 interface cards. 

Camera 72 can be controlled by a data I/O card in 
computer 90 , and light source 70 can be controlled by 
multi-functional I/O PWB with a dual D/A upgrade. 
Specifically, the data I/O card can be a MATROX 

15 INTERNATIONAL camera interface card (RS 422) . An 

associated processing accelerator card (described 
above) can be a MATROX INTERNATIONAL GENESIS 64/8 and 
MATROX INTERNATIONAL GPRO 16/8 processing card set that 
communicates via a high-speed backplane. With a MATROX 

2 0 INTERNATIONAL GENESIS card, additional MATROX 

INTERNATIONAL GPRO cards can be added up to 6 total to 
increase the processing capacity. Peripherals 92 can 
include a 17 inch color monitor, standard keyboard and 
mouse/trackball, and a 4 Gbyte DAT tape drive. The 

25 network interface (not shown in FIGURE 5) can be a 100 

Mbit (100 BASE T) Ethernet card. 

FIGURE 6 is a diagram of one embodiment of the 
imaging system of station 60. As shown, camera 72 is 
connected to optic 74 which includes an adapter optic 

30 94 and a telecentric optic 96. Telecentric optic 96 
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provides a support for mount 98 on which bifurcated 
line heads 76 are mounted. Heads 76 direct structured 
focused light on object (printed wiring board) 66 which 
is reflected and captured by optic 74 . 
5 In operation, station 60 of FIGURE 5 can provide 

automatic detection and classification of component, 
lead, and board level defects on populated printed 
wiring board 66, including discriminating between 
defects and "don't care" anomalies. Station 60 

10 provides rapid throughput with real-time optical 

inspection and, for example, can inspect an IB by 20 
inch board 66 with seven passes in approximately two 
minutes. Station 60 does not require on-the-floor 
programming to modify or add defects or to change board 

15 types, the latter only requiring standard component 

placement and data CAD files and board samples 
(populated and unpopulated) . 

Station 60 includes automatic background self- 
diagnostics and correction and can be networked with 

20 other stations or a central management unit. Further, 

station 60 can be set up to electronically transfer 
messages (e.g., e-mail, TCP/IP, paging, and other 
messages) as well as appropriate images for engineering 
analysis of low confidence classifications. 

25 Station 60 can be delivered fully functional to 

automatically detect and classify defects and can be 
trained to classify new defects encountered in 
production. For example, the table provided in 
APPENDIX A, below, shows defects that station 60 is 

3 0 trained to detect and classify. Further, APPENDIX B 
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provides example specifications for one implementation 
of station 60. 

The process implemented by inspection station 60 
can include a number of operational modes. At power- 
up/log- in, station 60 can automatically perform 
diagnostics and calibration tests and adjustments on 
critical inspection station parameters such as 
illumination, control and overall system health. An 
initialization window can then prompt the operator to 
select a board type from either a directory or to enter 
the name of a new board type. If a new board type is 
specified, station 60 enters a changeover mode. In the 
new board changeover mode, the station interface 
prompts the operator through the stages to create, 
develop, and load a new board knowledge base. If a 
known board type is specified, station 60 verifies that 
the complete knowledge base is available, and station 
60 is then ready to inspect. Station 60 generally can 
be ready for board inspection in minutes and returns 
the inspection and characterization window to the 
operator. Also, station 60 runs system diagnostics 
continuously in the background during operation. 
Automatic correction is made or the operator is 
prompted with the corrective action required. 

During inspection operation, station 60 typically 
allows two modes to be selected: characterization and 
inspection. In the characterization mode, station 60 
classifies board defects in greater detail and outputs 
complete information required for process control (PC) 
feedback. In inspection mode, station 60 acts in a 
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pass/fail mode, as a production line gate (generally, 
well understood and controlled processes) . Station 60 
can toggle between characterization and inspection 
modes. When station 60 encounters a defect in which it 
has low confidence for classification, it queries the 
operator for input and enters an incremental training 
mode. Upon completion of the incremental training, 
station 60 returns to the inspection/characterization 
mode and continues operation. 

In the incremental training mode, the addition (or 
removal) of defect classes to (or from) the knowledge 
base can be performed, assuming proper authorization. 
When station 60 classifies a defect in which it has low 
confidence, the station interface can show both the 
region in question and the defect in question on the 
display. Station 60 can then provide its best guess 
regarding the classification and offer the operator 
three choices: (1) verify that its best guess is indeed 
correct; (2) identify its best guess as wrong and 
classify the defect as another type previously known to 
the station; or (3) identify its best guess as wrong 
and classify the defect as a new type previously 
unknown to the station. Station 60 can ask the 
operator to verify or correct the classification and 
several example images and information files of each 
defect class. or prototype relevant to a classification 
decision can be presented to the operator to aid in the 
decision. Optionally, a line manager, engineer, or 
technician can be the only one authorized to make these 
decisions. If station 60 is networked, the results of 
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incremental training can be broadcast to a central 
control and/or directly to all other networked 
stations . 

Station 60 has a report mode in which summaries of 
the results of inspection/characterization and station 
status can be generated. For example, the total number 
of a certain type of defect over the last N boards can 
be graphically presented or a report of the station's 
calibration history can be generated. Reports can be 
automatically produced and electronically distributed 
on a regular schedule. Further, an analysis mode 
allows trends in inspection data to be used for process 
understanding and control. The analysis (or data- 
mining) mode of station 60 allows the operator to 
determine the relationships between process parameters 
and product quality (defects) over a range of time 
scales. Station 60 can thus play a crucial element in 
process control. Early detection and classification of 
defects alerts the operator to problems and allows 
early correction at the malfunctioning manufacturing 
stage . 

The inspection station of the present invention 
could be implemented using numerous alternative 
components other than those shown in FIGURE 5 . 
However, certain components have been selected as being 
well suited to the task of inspecting populated printed 
wiring boards. For the imager, a line scan camera is 
particularly useful because it gives higher resolution 
imagery over a wider field of view than can be obtained 
from available, low-cost area cameras. A line scan 
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camera is made. to continuously scan an image, so it can 
work with moving parts which naturally complements the 
inspection process because that is how objects are 
typically presented. Ideally, the line scan camera can 
5 provide a 100% resolution, or 100% fill factor, whereas 

an area camera will not. The line scan camera can 
accomplish this over a larger field of view and can 
collect data faster on moving targets. An additional 
aspect of the selected camera is that the camera is a 

10 time delay integration (TDI) line scan camera. The 

reason for choosing a TDI camera is that it overcomes 
the inherent light insensitivity of conventional line 
scan cameras. The TDI line scan camera provides an 
ability to get high speed imagery without needing 

15 excessive amounts of light. Without the TDI aspect, an 

excessively bright light source might be required. The 
TDI line scan camera allows low light levels to be used 
while the camera is run at a relatively fast rate 
(e.g., 8,000 lines a second at low illumination). 

20 A telecentric optic component is used with the 

camera to provide a "straight on" view of the board 
under inspection. In particular, a MELLES GRIOT 
INVARJTAR telecentric optic is used because it provides 
advantages over conventional optics and an advantage 

25 over other telecentric optics. First, conventional 

optics view part features at different angles, up to 3 0 
degrees or more, across the FOV. These changes in 
viewing perspectives can make the tops and bottoms of a 
given object appear at different x,y locations in the 

30 image, and can cause small components at the edges of 
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the FOV to be occluded by larger components that are 
closer to the center of the FOV. INVARITAR optics are 
designed to view objects "straight on" across the 
entire FOV, removing both limitations. Second, with 
5 conventional, and many telecentric optics, the 

magnification depends on the object -to-optic distance. 
Objects that are close to the camera are imaged larger 
than objects that are farther. INVARITAR telecentric 
optics provide large gauging depths of field (DOF) 

10 because their magnification is independent of object- 

to-optic distance. Compared to conventional, and most 
telecentric optics, INVARITAR telecentric optics can 
reduce magnification errors by a factor of 10 or more. 

The optic physically mounts directly to the camera 

15 and needs to match the camera so that the optic focuses 

the required FOV onto the camera sensor. So, if the 
sensor has sides X and Y, and the optic field of view 
is A and B, the optic needs to map the field of view 
down to the sensor (A,B:X,Y) . The calculation of FOV 

20 involves the resolution required and the number of 

pixels in the line scan array. For example, if a 1 
mil. (.001") image pixel resolution is required, a 
2,048 pixel line scan camera can image a 2.048" FOV. 
The calculation of the required optic magnification 

25 involves the physical size of the line scan imager 

pixels and the image pixel resolution. 

M = physical size/resolution = 13/zm/25 . 4/im 
image pixel resolution = .001" (25.4/im) 
imager size = 2,048 pixels 

30 FOV = (.001") - 2048 x (.001") • 1 
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I 2.048" 

.001" 

Physical pixel size = 13pm for CL-E2 

physical pixel size in image 

magnification required - ; — : : — : 

image pixel resolution 

5 

_ 13 pm _ 13/jm _ 13 
~ .001" ' 25.4pm " 25.4 

One implementation uses the following: 

Image pixel resolution 1.383 mils 35.14/zm 
Imager size 2,048 pixels 

10 =► 2.832" x .001383" FOV 

35.14pm 

If not for the capabilities provided by an optic 
such as the INVARITAR telecentric optic, the inspection 
of many small occluded parts and leads (which are 
almost always occluded by the part itself) would be 

15 virtually impossible because the required image data 

could not be obtained. Getting a wide field of view 
optic, with high resolution, is typically difficult to 
do. However, this is possible with, and is yet another 
advantage of the described INVARITAR telecentric optic. 

20 Additionally, the benefits of an optic such as the 

INVARITAR telecentric optic improve the overall 
accuracy and repeatability of all inspections. 
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With respect to illumination, there are numerous 
ways to accomplish illumination depending on what it is 
that needs to be imaged. On a printed wiring board, 
since there are many visible colors present, white 
light of the entire visible spectrum is used, and the 
light source needs to be as uniformly distributed as 
possible. Specifically, a halogen light source is used 
along with a structured light system with heads on each 
side that are driven by a bifurcated fiber optic cable 
from the source. The structured lighting provided by 
the heads points on the board and reflects up and 
collects on the sensor. Altogether, the imager, optic 
and illumination provide a top down view with minimized 
distortions and with a structured light focused at a 
certain point . 

The structured illumination provides significant 
advantages by allowing the generation of three- 
dimensional spatial information from a two-dimensional 
captured image. The third dimension (height) can be 
captured based upon the relative intensity of the 
reflected light and apriori information about the 
topology and reflectivity of the object being imaged. 
Things like solder paste volume can then be estimated 
from the three-dimensional spatial data. With the 
structured light, a three-dimensional surface (i.e., a 
board) is moved under the imaging system, a particular 
reflection back into the camera is obtained depending 
on the actual topology of that surface. So, for 
example, if the system is looking at a solder joint, 
the reflection will be noticeably different depending 
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on what portion of the joint is lighted and the 
physical structure of the joint (defective or not) . 

The intensity or the change in intensity provides 
information from which three-dimensional spatial 
features can be recovered- For example, the rate of 
change can be recorded as a gradient for the inspected 
surface. This allows equal height surfaces to be 
reconstructed for the object, somewhat like a topology 
map. Depending on the resolution, angle and structure 
of the light, a volume estimate can be generated from 
this information by initially calibrating based upon a 
known volume . The imaging system design enables this 
type of analysis and provides information that most 
systems can not capture. 

On the control and processing side, there are 
generally a number of printed circuit boards that can 
accomplish the various functions. For example, the 
imager feeds data to a data I/O board that then 
directly feeds a processing accelerator card. The 
imager has control signals that tell it to collect and 
send image data. The data is then collected and 
reconstructed into a two-dimensional image. The 
processing accelerator cards have a direct high-speed 
connection to the data I/O card. Additionally, data 
transfer between accelerator cards can occur across a 
backplane and not across the host system bus. Because 
the host system is involved with executing the final 
classification algorithms and graphical user interface 
(GUI) and managing the control signals, it would be 
undesirable to over task it with either pre-processing 
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or with data transfer. It is for this reason that 
intensive data transfer and processing is accomplished 
by the processing accelerators. The resulting 
descriptors are then passed to the host computer over 
the system bus. As for the physical platform, it is 
recognized that the platform may not be completely 
perfect. Thus, built-in mechanisms are included to 
allow adjustment. 

One particular feature supported by the host 
system is the ability to electronically transfer a 
message to a production manager so that the production 
manager does not have to come over to the manufacturing 
area each time something happens. The production 
manager can receive the electronic message in a remote 
office, look at any attached images and do what needs 
to be done for the classification. 

Inspection Process 

FIGURE 7 is a flow chart of one embodiment of an 
automated inspection process implemented using station 
60 of FIGURE 5. As shown, in step 100, the station 
acquires image data from scanning the board being 
inspected. The image data is then pre-processed in 
step 102. After pre-processing the image data, the 
image is described by selected descriptors in step 104. 
The descriptors are then analyzed, in step 106, such 
that the image is analyzed in context with the system 
knowledge base. In step 108, defects are identified 
and classified. 
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In step 110, the station determines whether the 
confidence in the classification is above a preset 
threshold. If so, the station checks, in step 112, 
whether analysis of the board is complete. If not, the 
process continues at step 100. If the analysis is 
complete, the process continues at step 114 with the 
ejection of the board and obtaining a new board to 
inspect . 

If, in step 110, confidence was not above the 
threshold, then the process continues at step 116. In 
step 116, the operator is alerted and provided with the 
station's best option as to the proper classification. 
In step 118, the station accepts confirmation of the 
best option by the operator or accepts an alternate 
classification by the operator. Then, in step 120, the 
station adds information to the knowledge base as 
necessary to accomplish incremental learning based upon 
the operator's input. The process then continues at 
step 112 as discussed above. 

Classification is accomplished according to the 
present invention in accordance with a set of pre- 
specified policies. Descriptors are extracted and 
stored to the knowledge base. Then, when a new image 
is acquired and a set of descriptors are generated from 
it, the new descriptors are compared by a variety of 
metrics to those stored in the knowledge base to 
determine whether or not a defect exists and what type 
it is. So, if there is a good match, the system can 
have confidence that the object being inspected 
contains the specified defect. If there is not a good 
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match, the system can select the closest match as a 
best option. The operator can then provide feedback 
as to whether the option is accurate, a different class 
should be selected, or a new class should be defined. 
If it is the latter, then in one step, an entirely new 
class of defects is established. In this manner, the 
knowledge base is built incrementally from a base level 
of information to include more and more knowledge. In 
addition, the system generates more and more trust with 
the operator because the operator sees his decisions 
appear in the decision making of the system. In fact, 
part of the incremental learning cycle can be not only 
to store the descriptors but also to store who made the 
decisions on low confidence classifications and when. 
This allows decisions to be back tracked out of the 
system if they prove to be poor judgments. The 
confidence level of the system can also be based upon 
how many of the particular type of defect the system 
has seen. Thus, the confidence level represents not 
only how confident the system is in the single 
classification but also populations of classifications. 
The statistics therefore use populations of information 
with known quality. 

Process control can then be based upon the 
classifications. By classifying defects and 
maintaining a history, the overall process can be 
improved by analyzing defect histories and matching 
defects to a cause. Placing decisions as to how to 
classify a defect in context with other defects and 
decisions produces higher quality input from operators, 
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and over time, _the knowledge base gets more and more 
refined. 

The implementations described herein are not 
intended to limit the scope of the present invention. 
For example, in addition to populated printed wiring 
boards, the present invention is applicable to analysis 
of bare printed wiring boards, semiconductor packages 
(e.g., PPGAs, PBGAs, C4 packages, or TAB tape) and 
other types of objects. Also, in contrast with the 
described implementation, the imaging system can be 
fixed while the object to be analyzed is moved. This 
approach would allow the imaging system to remain more 
stable during scanning which would allow more precise 
motion for ultra high resolution images (e.g. , 
pixels vs. 30^m pixels), but would likely require a 
larger platform with a wider range of movement. 

Further, the inspection system can be integrated 
with other pieces equipment rather than being separate. 
For example, an integrated piece of equipment can 
include both object processing components and automated 
visual inspection components. The object processing 
components receive the object and process it to prepare 
it for subsequent steps in the manufacturing process. 
The automated visual inspection components perform 
substantially as described above to identify any 
defects after the object has been processed. Then 
control components can be used to instruct the object 
processing components to correct the defect prior to 
passing the object on to the subsequent step in the 
manufacturing process. The object, for example, can be 
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a printed wiring board where the object processing 
components perform solder paste deposition, and the 
automated inspection components allows solder paste 
verification. Another example is that the object 
processing components can perform pick and place of 
components, and the automated inspection components 
allow component placement verification. In this 
manner, integration with processing equipment allows 
the equipment to ensure objects are complete before 
being passed downstream in the manufacturing process. 

Although the present invention has been described 
in detail, it should be understood that various 
changes, substitutions, and alterations can be made 
hereto without departing from the spirit and scope of 
the invention as defined by the appended claims. 
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APPENDIX A 
TABLE OF DEFECTS 



Defect Type 


Defect Description 


Defect Detection 6 
Classification (DDC) 
Package 


Board defect 


• 


Warped/twisted panel 


Board Verification 




• 


Cracked/broken card 


n 




• 


Scorching 


A 




• 


Scratches 


n 




• 


Breakaway 


t» 




• 


Burrs 


u 




• 


Lasermark - missing, illegible, 
incomplete , incorrect , 
misoriented, misplaced, 
blisters 


Code Reader & Data 
Logging 




• Surface Foreign Material 


Board Verification 


Component defect 


• Wrong component 


Component ID 




• Mis oriented component 


Component 




• Missing component 


n 




• Component body void/pit /indent 


n 




• Damaged component 


o 




• Heat slug void 


tt 




• Component stain 


a 




• Component foreign material (FM) 


a 




• Component misalignment 


it 




• Component s)cid 


n 




• Tombstone condition 


R 




• Missing lead 


Lead Quality 




• Bent lead 


n 




• Lifted lead 






• Peeling cracked termination 


n 




• Lead damage 


D 




• Board misregistration 


Component 
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Defect Type 


Defect Description 


Defect Detection & 
Classification (DDC) 
Package 


Solder joint 
defect 


• Insufficient solder - poor/non- 

wetting of lands or leads, 
dewet t ing 


Solder Joint Quality 


• Fractured solder joint 


m 


• Excess solder 


tt 


• Solder bridging 


ti 


• Solder balls 


it 


* Colder solder 


n 


• Solder projection 


it 


Gold finger 


• Foreign material on finger (s) 


Board Verif ication/PWB 






• Surface damage to f inger ( s ) 


o 


• Finger nodules 


a 


• Finger void 


n 


• Solder on finger 


n 


Connector defects 


• Blade missing 


Connector & Add -On 


• Blade misalignment 


n 


• Connector body chipped, cracked or 
broken 


a 


• Foreign material on connector body 


B 


• Connector body misalignment 


n 


• Solder on blade 


it 


Solder paste 
defects 


• Coverage XY - too little/much 


Solder Paste 


• Volume XYZ - too little/much 


n 


• General quality 
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APPENDIX B 

SAMPLE STATION FEATURES AND SPECIFICATION 

Standard features 

• Can be configured for On-Line or Off-Line 
5 application. 

Material handling system is a SMEMA 
compatible conveyor segment and is compatible 
with other SMEMA production line equipment . 
On-Line Configuration: Station is integrated 
10 into the production line under SMEMA control. 

Off -Line Configuration: Station is not 
integrated into the production. The station 
is loaded/unloaded manually with conveyor 
assist . 

15 • Real-time 100% inspection at production line 

rates. 

Scalable processing engine allows increased 
line rates to be met. 

Processor architecture is transparent to the 
20 operator. 

• Accommodates complex, high-density board types. 

High-resolution, high-quality, line-scan 
imagery. 

100% board coverage without part occlusions. 
25 • Real-time, on-line learning. Point-and-click: 

Addition of new defect classes (seconds) . 
Correction of incorrect classifications 
(seconds) . 

Re-enforcement of correct low- confidence 
30 decisions (seconds) . 

Board change over (new board - 
minutes/pret rained board - seconds) . 
Inspection routine modification (seconds - 
minutes) . 

35 • Low escape and false-alarm rates. 

Embedded classification analysis mechanisms 
(confidence and relative confidence) flag 
questionable classifications for operator 
analysis and immediate correction which 

40 pushes escape rates down. 

On-line learning allows rapid tuning of 
marginal defect limits which pushes false- 
alarm rates down. 

• Low operator skill requirements. 
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Point -and-Click Graphical User Interface 
(GUI) . 

Delivered with a functional knowledge based 
installed (turn-key operation) . 
5 - Automated new-board changeover routines 

(including inspection routine development) . 
New board change over takes only minutes. 
Automated calibration and calibration 
verification routines. 
10 • A broad range of defect detection & classification 

(DDC) software packages including: 
DEC PescriptiQtt 

1 Component Package 

2 Code Reader & Data Logging Package 
15 3 Lead Quality Package 

4 Solder Joint Quality Package 

5 Connector & Add-On Package 

6 Label & Marking Package 

7 Component ID Package 

20 8 Board Verification Package 

9 PWB Package 

10 Solder Paste Package 

[Refer to APPENDIX A for an example of a detailed 
listing of the defects detected and classified by the 
25 different station DDC packages.] 

• Real-time generation of process control 
information and reports. 

Assessment of production-process and product 
quality. 

3 0 - Aids in defect source identification. 

Real-time input to SPC packages. 

Specifi cations 

Board Size: 2 0" x 18" x ±2" (L x W x H - max.) , 

35 3" x 3" (L x W - min. ) . 

Throughput: Throughput is primarily a function 

of component count and inspection 
routines being applied. Board area 

40 does not typically affect the 

station throughput; only for very 
sparse boards does board area set 
throughput levels. The throughput 
numbers provided assume a complete 

45 set of relevant inspection routines 
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is applied to each part. Minimum 
part size: 0402. Minimum lead 
size: 10 mil pitch. 

100 components (average mix) per 5 
seconds nominal throughput. 

[Processing engine upgrades provide 
higher throughput] 

(ex. a 2000 component 18" x 20" 
board- ->1 00 seconds +20 seconds 
material handling and data 
collection overhead --> 120 seconds 
or 2 minutes) 



15 



Footprint : 



20 



Automation: 



25 



30 



GUI Requirements : 



35 



40 



1m (L) x lm (W) x 2m (H) . Compact 
design occupies the production line 
floor space required by a standard 
lm conveyor section. Allows for 
On-Line and Off -Line application. 
Manual assist (off line) or 
conveyor hand- off (on line) . For 
off-line use, the conveyor 
positions a board, fed by the 
operator, for inspection and then 
returns the board to the operator 
after inspection is complete . For 
on-line use, the conveyor positions 
a board, fed by the previous stage 
in the line, for inspection and 
then hands it off to the subsequent 
stage in the line after inspection 
is complete. 

GUI includes routines and screens 
for system start up, known board 
preparation, unknown board 
preparation, inspection, low 
confidence/defect verification and 
validation, new defect entry, 
knowledge base 

verification/editing, report 
generation, data base set up. 



45 



Data Archiving: 



Station includes network 
connection, disk drive and resident 
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2G hard drive for data storage and 
archiving. High-capacity tape 
drive systems are optional. 



10 



Facility 
Requirements : 

Dry. 



Safety Mechanisms : 



15 



20 



Maintenance/ 
Calibration: 



25 



Temperature Range: 70° - 85°F. 

Power: 110V, 60Hz, 20 Amps. 

Air: 100 PSI, 1.0 CFM, Filtered & 

Network Access at the Station. 

Emergency power-off button (entire 
station is shut down) and emergency 
interrupt button (only moving 
components are shut down) . 
Emergency interrupts are also 
generated when any user openable 
access panel is opened. Once 
triggered, a reset mechanism must 
be enabled to re-establish power. 

The Calibration Toolkit contains 
routines for automatic station 
calibration and calibration 
verification including illumination 
verification and adjustment, new- 
bulb calibration, station alignment 
and motion analysis, image quality 
assurance . 



30 



35 



40 



45 



The only regular station 
maintenance required is the 
periodic replacement of 
illumination system light bulbs. 
The station will automatically 
notify the operator when a given 
bulb set requires replacement . A 
step by step bulb replacement 
procedure will be provided to the 
operator through the GUI (no tools 
will be required). Once the bulbs, 
have been replaced, the station 
automatically performs calibration 
and returns to the inspection mode 
of operation. It is expected that 
the light bulbs will require 
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replacement no more than once every 
two months. It is recommended that 
two sets of spare bulbs be kept 
available for each station. 



Customer Data: 



10 



To ensure a "knowledge -full" 
knowledge base at delivery, it is 
requested that 30 samples of each 
defect type be provided prior to 
station delivery. 



Options : 



15 



Packages . 



Flat Panel Display. 
Touch Screen Display. 
Networking . 
Tape Drive. 
Printer . 

Throughput Accelerator 
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WHAT IS CLAIMED IS : 

1. An automated visual inspection system, 
comprising: 

an imaging system operable to view an object and 
to provide image data representing an image of the 
ob j ect ; 

a precision positioning system coupled to the 
imaging system, the precision positioning system 
operable to move the imaging system to scan the object, 
and the precision positioning system operating 
responsive to control signals; and 

a processing and control engine coupled to receive 
the image data from the imaging system and to provide 
control signals to the precision positioning system, 
the processing and control engine operating: 

to acquire and pre-process the image data; 

to represent the image of the object using 
descriptors ; 

to compare the descriptors to information in 
a knowledge base and to identify and classify a defect 
based upon results of the comparison; and 

to associate a confidence level with the 
classification of the defect. 

2 . The automated visual inspection system of 
Claim 1, wherein the processing and control engine 
further operates, if the confidence level is not above 
a specified threshold: 

to alert an operator and provide a best option as 
to the classification of the defect; 
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to accept a confirmation or alternate 
classification from the operator; and 

to add information to the knowledge base 
responsive to the confirmation or alternate 
5 classification. 

3 . The automated visual inspection system of 
Claim 1, wherein: 

the imaging system comprises a structured, focused 
10 light; 

the imaging system operates to provide reflected 
light intensity as part of the images data; and 

the processing and control engine operates to 
recover three-dimensional spatial information based 
15 upon the reflected light intensity when the image data 

is pre-processed. 

4. The automated visual inspection system of 
Claim 1, wherein the imaging system comprises: 

20 an illumination system providing light to 

illuminate the object ; 

an optic to gather an image of the object; and 
an imager to sense the image and generate the 

image data. 

25 

5. The automated visual inspection system of 
Claim 1, wherein the object is coupled to a material 
handling system that operates to move the object to a 
scanning range of the imaging system. 



30 
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6. The automated visual inspection system of 
Claim 1, wherein the processing and control engine 
comprises : 

a control hub that receives status information 
5 from and sends control signals to the imaging system 

and the precision positioning system; 

a data input /output device that receives the image 

data; 

processing accelerators that pre-process the image 
10 data and generate the descriptors; and 

a software engine executed by a host computer that 
manages processing and control, accomplishes the 
identification and classification and drives a 
graphical user interface. 



15 



20 



7. The automated visual inspection system of 
Claim 1, wherein the processing and control engine 
further operates to share knowledge base information 
with other inspection systems. 



8 . The automated visual inspection system of 
Claim 1 # wherein the processing and control engine 
further operates to send an electronic message to a 
production manager, the electronic message including 
25 the best option as to the classification of the defect 

and the descriptors. 



9. .The automated visual inspection system of 
Claim 1, wherein the processing and control engine 
30 further operates to send an electronic message with 
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status and inspection reports to specified groups at 
specified intervals. 

10. The automated visual inspection system of 
Claim 1, wherein the object is a populated printed 
wiring board. 

11. An automated visual inspection station for 
inspecting populated wiring boards, comprising: 

a platform providing physical support for the 
inspection station; 

a pair of guides coupled to the platform, the 
guides providing a path for a board to be inspected; 

a stop coupled to the platform, the stop operable 
to position the board for inspection; 

a thumper coupled to one of the guides, the 
thumper operable to position the board against the 
other guide; 

an optic operable to obtain an image of the board; 

a camera coupled to the optic, the camera operable 
to acquire the image of the board and provide image 
data; 

a light source operable to provide light for 
illuminating the board; 

a bifurcated optical fiber coupled to the light 
source ; 

a pair of linear heads coupled to receive light 
from the bifurcated end of the optical fiber and 
coupled to the optic, the heads operable to illuminate 
the board with structured, directed light; 
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a first precision linear guide and first motor 
coupled to the camera and operable to control movement, 
along one axis, of the camera, optic and heads to scan 
the board; 

a second precision linear guide and second motor 
coupled to the platform and to the first precision 
linear guide and operable to control movement, along a 
second axis, of the camera, optic and heads to scan the 
board; 

interface devices coupled to provide an interface 
to the stop, the thumper, the camera, the light source, 
the first motor and the second motor; and 

a processing and control engine coupled to the 
interface devices, the processing and control engine 
operable to receive the image data and to provide 
control signals to the stop, the thumper, the camera, 
the light source, the first motor and the second motor, 
and the processing and control engine operable to 
identify and classify a defect based upon analysis of 
the image data. 

12. The automated visual inspection station of 
Claim 11, further comprising peripherals coupled to the 
processing and control engine. 

13 . The automated visual inspection station of 
Claim 11, wherein the processing and control engine is 
further operable to use descriptors to represent the 
image data and to compare the descriptors to 
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information in a knowledge base to identify and 
classify the defect . 

14 . An integrated piece of equipment for 
completing a step of a manufacturing process, the 
equipment comprising : 

object processing components operable to receive 
an object and to process the object to prepare the 
object for a subsequent step in the manufacturing 
process; and 

automated visual inspection components operable: 

to acquire image data representing an image 
of the object ; 

to pre-process the image data; 

to represent the image of the object using 
descriptors ; 

to compare the descriptors to information in 
a knowledge base; and 

to identify a defect based upon results of 
the comparison. 

15. The integrated piece of equipment of Claim 
14 , further comprising control components operable to 
instruct the object processing components to correct 
the defect identified by the automated visual 
inspection components prior to passing the object on to 
the subsequent step in the manufacturing process. 

16. The integrated piece of equipment of Claim 
14, wherein the object is a printed wiring board, the 
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object processing components perform solder paste 
deposition, and the automated inspection components 
allow solder paste verification. 

17 . The integrated piece of equipment of Claim 
14, wherein the object is a printed wiring board, the 
object processing components perform pick and place of 
components, and the automated inspection components 
allow component placement verification. 

18. An automated visual inspection process, 
comprising: 

acquiring image data representing an image of an 
object ; 

pre-processing the image data; 

representing the image of the object using 
descriptors; 

comparing the descriptors to information in a 
knowledge base; 

identifying and classifying a defect based upon 
results of the comparison; and 

associating a confidence level with the 
classification of the defect. 

19. The process of Claim 18, further comprising, 
if the confidence level is not above a specified 
threshold : 

alerting an operator and providing a best option 
as to the classification of the defect; 
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accepting a confirmation or alternate 
classification from the operator; and 

adding information to the knowledge base 
responsive to the confirmation or alternate 
classification . 

20. The process of Claim 18, further comprising: 
providing reflected light intensity as part of the 

images data; and 

recovering three-dimensional spatial information 
based upon the reflected light intensity when the image 
data is pre-processed. 

21. The process of Claim 18, further comprising 
sharing knowledge base information with other 
inspection processes. 

22. The process of Claim 18, further comprising 
sending an electronic message to a production manager, 
the electronic message including the best option as to 
the classification of the defect and the descriptors. 

23. The process of Claim 18, wherein the object 
is a populated printed wiring board. 

24. The process of Claim 18, further comprising 
performing automatic calibration and correction 
algorithms . 
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25. The process of Claim 24, wherein performing 
comprises : 

assisting a user to adjust adjustable alignments; 

automatically computing relevant system 
5 misalignments and distortions and generating correction 

coefficients that are applied during inspection; and 

checking lights before each inspection cycle and 
automatically performing any required adjustments and 
calculations . 
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